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Abstract 



ASSESSMENT IN MASS COMMUNICATION 



Departments of mass communication employ about 15 different types of assessment to 
measure either their students' individual achievement or the curriculum and teaching. Of these, exit 
examinations for seniors have raised the most apprehension. This paper reports the results of a 
national survey of 276 four-year colleges and universities teaching mediated communication under 
such departmental titles as mass communication, radio-television, broadcasting, telecommunications, 
and mass media. Results (response rate exceeded 82 percent) showed that 94 percent of media 
departments do not use exit tests, and most of the 6 percent that do, use them to measure 
improvements in curriculum and teaching, not to measure individual student achievement And even 
when individual student outcomes are the goal, few schools make pass exit tests a condition of 
graduation. Case reports based on interviews with the 6 percent of schools now using exit tests 
illustrate the range and type of objective-style and essay-style exit testing. The paper concludes with 
a ten-part assessment decision-making model. 



ASSESSMENT IN MASS COMMUNICATION 



Educators are living in the Age of Assessment. The idea of assessing achievement — in the 
sense of measuring, comparing, ranking, judging — schools, teachers, curricula, and teaching 
methods is an outgrowth of the philosophy of accountability} Increasingly, both government and 
the public are trying to find ways to hold the recipients of public funds responsible for what they 
do with that money. In the case of education, governme'iiTofficiaTs and the public want assurance 
that students are indeed being educated. No longer does a college degree automatically guarantee 
employment and high earnings. Too many cases of high-school graduates who cannot read and 
college graduates (even Vice Presidents) who cannot spell have waved a red flag at the bull of 
education. Concern about the quality of undergraduate teaching in light of rapidly increasing tuition 
costs is forcing universities and colleges to justify themselves. The public now mistrusts education 
and educators, and, in response, governments seek to hold colleges and universities accountable. 
They want a warranty on education. 

Higher education's accreditation bodies are also getting into the act by demanding that 
universities document their students' academic achievement as a criterion for accreditation. The 
North Central Association, for example, requires universities to have "clear and publicly stated 
purposes," show how they've organized their resources to achieve these purposes, and demonstrate 
ongoing progress in accomplishing these purposes ("Mandated Assessment," Academe, 1990). Such 
general requirements can be met through documentation of student and faculty recruitment and 
retention practices, financial audits and budgeting procedures, safety and space allocations, library 
acquisitions, and, finally, academic degree program viability. Accrediting agencies have traditionally 
been more directly concerned with the addition, reduction, or elimination of departments and 
programs than with student outcomes achievement, though indirectly, all these factors affect how 
well the faculty teaches. However, beginning in the 1990s, accrediting associations turned their 
attention to assessment in general education and the major. 



In theory, more or better assessment could improve both teaching and learning. Assessing 
what is achieved can improve curricula, motivate teachers, and motivate students. The question often 
becomes what type of assessment when state legislators or university administrative fiat removes the 
choice of "no more." In spite of persuasive arguments from prestigious educational associations, 
among them the American Association of University Professors, in favor of multiple assessment 
measures rather than standardized outcomes testing ("Mandated Assessment,^ 1990), states such as 
New Jersey, Oklahoma, Tennessee, and Illinois have mandated quantifiable outcomes testing for 
general education and in the major, and Texas, Colorado, Virginia, Missouri, Louisiana, and 
Kentucky have instituted economic or accreditation pressures-';'/':th s-i-n^.iiar effects. Tennessee, for 
example, has tied a portion of higher education funding (called "^performance funding") directly to 
explicit demands for quantitative tests in major fields. The Tennessee Higher Education Commission 
(1992) has specified that campuses may use either an externally developed test (from an approved 
list) or a test developed locally in accordance with its guidelines. Florida, Georgia, and South Dakota 
also have established statewide assessment. Several of these states have espoused the "valued added" 
approach, requiring state-supported colleges and universities to demonstrate how much their students 
have gained since entering college. However, Boyer et al. (1987) point out that "most of the newer 
statewide programs, on the other hand [excepting Tennessee], are explicitly avoiding the "rising 
junior" ov "value-added" approaches typified by these early entrants." 

Objections to such outcomes testing are legion. Nielsen and Polishook (1990), for example, 
claim: "Despite their superficial appeal, such schemes are as likely to harm as enhance the educational 
process, and their results are as likely to confuse as inform" (p. A 14). Students do not start with the 
same knowledge, ability, and skills, never have the same formal and informal learning experiences 
while in school, and gain different things even from the same lessons. Students differ too much from 
one another and what they lea'-n is too diffuse and too diverse to ever be captured meaningfully in 
standardized tests. Such tests seem to measure mostly one's ability to perform well on future similar 
tests, the traditional (but unloved) fare of college course examinations. Thus, proposals to institute 
standardized pretests and posttests to measure what they learned in school look like sieves. Many 
teachers contemplate more standardized testing with despair. They justifiably complain such tests 
fail to measure much of anything worth measuring about their students (Eastman, 1987). Goals such 
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as developing cognitive skills, adding to intellectual capacity, developing creative capacity, and 
enhancing depth of appreciation and understanding lie beyond quantitative measurement (Nielsen 
& Polishook, 1990). 

Ironically, along with calls for assessment, faculty can hear the clarion call for improving our 
schools, in part by moving away from standardized tests of facts and definitions toward focus on 
qualitative and affective learning and increased application of concepts and skills. The dangers of 
"teaching to the test," fostering intellectual conformity, and the invitation to concentrate on the trivial 
while devaluating the controversial and unmeasurable have been widely touted. Indeed, the challenge 
of multiculturalism moves colleges and universities further away from ccre cur-ricula and thus further 
from common bodies of knowledge about which to test (Chase, 1990). 

Some objections to outcomes testing may recede if testing efforts are focused only on 
cumulative learning in the major field of study. This paper summarizes the key issues and describes 
the types of outcomes assessment in the major appropriate to the field of mass communication. Then 
it reports the results of a nationwide study of electronic communication departments on the subject 
of exit examinations, analyzes the tests that some colleges and universities now use, and finally 
proposes a model of what can and cannot be effectively achieved in this arena by media departments. 

Assessment in the Media Major 

Experts in the assessment field conclude that small liberal arts colleges can effectively tackle 
across-the-board assessment in general education, whereas large research universities should 
undertake as:essment at the departmental level (Magner, 1989). However, according to the 
bibliography maintained by the Assessment Resource Center at the University of Tennessee, more 
than 40 fields lack national exams, including, to no one's surprise, masss communication and 
journalism. 

In his 1988 study of journalism programs, R. Ferrell Ervin summarized the advantages of 
national testing instruments for the major as (I) saving faculty time, (2) providing norms for 
comparison, and (3) theoretically resolving questions of credibility and validity (although ethers, such 



as Baker, 1986, argue that point). He contrasts the advantages of local instruments as (1) reflecting 
the local curriculum, (2) serving as a valuable tool for planning and curricular revision, and (3) 
involving faculty input (and by implication, their support). However, Ervin lays out the contrasting 
disadvantages of objective and essay tests in Table 1 , clearly showing that either option demands time 
and resources, commodities in short supply in media departments. 

Table 1 about here 

Function and Costs - — — 

Carol Schneider, Executive Vice President of the Association of American Colleges, 
efficiently points to faculty's two main conccirns: 

Faculty members in general are suspicious of and resistant to efforts 
to develop cumulative assessments of students' learning in the major, 
partly because they are wary about the uses to which assessment findings 
will be put, especially in state systems, and partly because of the 
predictable costs in faculty time and energy. (Quote^l in Wolf, 1989, p. 8) 

At publicly-supported universities, the very real danger arises of having quantitative student 
assessments twisted into measures of teaching achievement and thus affecting retention, promotion, 
and salary-setting.^ At universities adopting responsibility-center management and budgeting 
(RCM/RCB), the threat of financial impact becomes particularly critical if the department resides 
in an arts and sciences college rather than a school or college of communication.^ 

Furthermore, locally devised instruments are extremely time-consuming as they usually 
require psychometric expertise and extensive pilot testing, presuming that the faculty could agree 
on what to test for. For large universities, the process from start-up at the universities level to initial 
data collection at the departmental level is estimated at 3-5 years. Too often departments have 
rushed into commitments entailing exit exams without allowing for the years that proper pilot testing 
necessitates and without the expertise in psychometrics that developing recognized reliability and 
validity measures requires. Of course, expertise can be hired, but repeated testing of several classes 
of graduating seniors cannot be rushed. 

Some hair-raising data are available on the costs of cumulative testing. These costs include 
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the measurement instrument, its administration (pn. -toring if needed), the statistical analysis of 
results (and sometimes data entry), and coordination or > ripervislon of assessment activities. In mass 
communication departments, which are small compared to many traditional university departments 
such as History, English, and Biology, administration, analysis, and coordination probably can be 
handled by graduate students and/or faculty. This usually will involve a budget for hiring the 
students or released time for the faculty. But the enormous cost lies in developing locally-specific 
examinations. They are estimated to require over $3,000 per exam (Wolf, 1989), an amount far 
beyond the reach of many departmental budgets. 

Defining the Mass Communication Field . ~ 

One problem arises in considering departments of journalism and departments of 
communication in the same breath.* The former tend toward applied professional education; the 
latter contain some with the same goal (in, say, television production) and many others with more 
in common with departments of humanities or social science and little direct concern with 
"professional" job preparation (see Eastman, 1987; Gomery, 1985; Limberg, 1987; Sterling, 1985; 
Webster, 1989). Although professional journalists have loudly criticized the way colleges and 
universities prepare future journalists (consider the 1987 Roper Report), no such constituency exits 
for the broader field of electronic communication and its fellows, radio-television, 
telecommunications, and so on. 

Ervin found that 39 percent of journalism schools responding to his 1988 survey utilized 
outcomes assessment, but their definitions of what was appropriate ranged from ordinary assignments 
and grades to course evaluations concerned with the worth of the instructor and course to, in a very 
few cases, standardized exit testing.^ In the late 1980s, a few media departments experimented with 
the ACT, the Cooperative English Exam (COOP), or the general portion of the GRE, but recognized 
their inappropriateness for measuring major achievement in broadcasting, journalism, radio- 
television, telecommunications, and so on. 
Defining the Content 

Finally, but perhaps most significant, is the issue of what to test on within the major. The 
interdisciplinary field of mass communication subdivides itself into departments focusing on 
production and application, departments focusing on business and management careers, departments 



focusing on critical studies and aesthetics, departments emphasizing broadcasting to the exclusion 
of alternate technologies or, conversely, focusing on telecommunications while downplaying 
traditional broadcasting.^ Only a few telecommunications departments give telephone more than lip 
service. Furthermore, the field encompasses departments of journalism with a few mass 
communication courses and departments excluding all journalism. The range in size of schools, 
number of faculty, number of students, and number and kinds of courses is almost beyond counting 
(Robinson & Kamalipour, 1991). A single national test seems impossibly visionary. 

But even on the local level, students at a large university have programs within the major as 
diverse as the overall departments already described. Indiana University, for example^ div-ides-i-ts- 
undergraduate offerings into the four areas of Industry & Management, Electronic Media Production 
& Design, Telecommunications Technologies, and Society & Culture, but permits as many as a dozen 
specialties within the broad classifications to accommodate differing student goals and abilities. 
Some students have well-defined job goals; others are uncertain or concerned more with breadth of 
education than immediate employment. Moreover, many students graduate without wholly fulfilling 
any "track" as specific classes may be unavailable, overenrolled, or conflicting in scheduled time, and 
following a plan is voluntary anyway. Thus, devising a test to measure the learning of such diverse 
seniors on more than the most superficial level seems impractical. 

Moreover, "information" in the media field doesn't hold still. The key facts and concepts 
for students in 1983 are no longer quite so important (consider reregulation of broadcasting) or may 
even be reversed (consider the industry's reversal from the principle of scarcity necessitating program 
parsimony to the expectation of immense channel capacity as a result of digital compression). Even 
the main entities in the field are not now who they were a decade ago. Repeated revolutionary 
transformations of assumptions, theories, and practices so characterizes the fundamentals of 
electronic media that the experiences of such disciplines as History, English, and sciences provide 
little guidance. As long ago as 1980, textbook author F. Leslie Smith bemoaned recent quick changes 
in broadcasting; think how much more sweepingly the industry has changed and continues to change 
in the 1990s. 

But there is a worse problem for programs that position themselves within the liberal arts. 
Benjamin (1990) points out, "[Testing in the major] would encourage students' tendency toward 



excessive specialization and vocationalism and diminish the opportunity the major provides for 
independent and analytical inquiry" (p. Bl). Defining specialty tests in tracks within the major is 
likely to foster emphasis on entry-level job opportunities and devalue nonvocational liberal arts 
studies. 
Ethical Risks 

Based on a 1989 survey of 91 schools with membership in the Association of Schools of 
Journalism and Mass Communication for Education (ASJMC), Eshelman outlined five ethical dangers 
associated with exit testing in a 1991 presentation. Indeed, he indicted outcomes assessment because 
of the risk of creating untenable situations for faculty and administrators, and his conclusions apply- 
to a wide range of major departments. 

The first ethical risk is that departments can be coerced into adopting meaningless tests 
lacking all educational validity in response to political pressure. One such test Eshelman described 
consists of 25 questions from each of several mass communication courses; students are not required 
to pass the test and have no incentive to study for it. He also makes the second point that 
standardized tests cannot measure the creative skills that are essential parts of the artistic and craft 
cor.iponents of journalism and broadcast education, and thus most exit testing measures little of 
value, although departments must, in the academic way of things, make claims for the validity of 
tests. 

Eshelman goes on to make the third point that the answers to the questions used in many 
standardized tests often can be bought on the street, reporting examples from major newspaper 
investigations of test buying. Eshelman's fourth ethical criticism relates to "teaching to the test" (p. 
20), the strategy of teaching students what they need to know to pass. A fifth danger occurs when 
departments are pressured to show improvement in average scores; tests can be "refined" by 
eliminating difficult questions and substitute easier ones, thus raising average scores as desired. 
None of these practices is educationally and morally sound, and compelling or coercing departments 
into exit testing reflects administrators' and legislators' misunderstandings about the nature of the 
learning process and the goals of undergraduate education. 
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Faculty Resistance 

Having more than a small inkling of these concerns, most faculty, especially in 
interdisciplinary fields such as communication, initially resist the idea of exit examinations and 
often succeed in postponing serious developmental efforts for many years, even when such testing 
is mandated at the state or university level. Ervin (1988) quotes one administrator's candid response: 
"...these standardized tests provide easy scoring but the questions asked did not report how much his 
graduates had learned, but the GRE satisfied the Provost; if it satisfies his interest, we don't want 
to rock the boat'" (p. 22). However, it should be noted that Schneider (quoted in Wolf, 1989) 

^concludes that "faculty members who participate in assessments of students' cumulative learning are 

frequently both enlightened and engaged by the findings" (p. 8), Whether such a conclusion is 
warranted for media-related fields is unknown. 

Some kind of assessment in the major, however, is mandated in many states, by accrediting 
associations, and by some universities. Moreover, it can bring rewards in terms of reallocation of 
university resources and reaccreditation. For those with sincere commitment to teaching, one key 
purpose for assessing students' cumulative learning is to find out whether content, skills, and 
attitudes are indeed integrated, an explicit or implicit goal of most major programs. Moreover, 
assessment, at its best, can show how the learning process can be enhanced or streamlined (Wolf, 
1989). 

Types of Outcomes Assessment 

In addition to overall program review, nearly a dozen kinds of outcome assessment within 
the mass communication major can be identified. Many of these focus on knowledge of content, 
but they are largely indirect measurements, test only in one narrow arena, or are not suited to 
administration at the pre-graduation level. 
Standardized Curricula 

One form of assessment within the major is standardizing the curriculum in order to give 
every student approximately the same academic preparation, presumably the best preparation that 



9 



educators can devisej Speakers at recent BEA conventions have argued for and against proposals 
for specific core courses to standardize the media field. For proponents, the argument turns on 
identification of fundamentals that all media students "should know"; on the negative side, opposing 
any national curriculum, the argument is that the goals and faculty resources of departments vary 
so widely that commonalities beyond the introductory course content (already embodied in several 
widely used textbooks) are impossible. 
Core Curriculum 

Leaving aside other schools, another form of standardized curricular design is the core 
^progcam within a single department. Some faculties identify concepts, theoretical arenas, and skills 
specific to the major that they require of all majors (Schwalbe, 1991; Burke et al., 1992). Having 
a core of courses required of all majors implies that this body of knowledge and skills is essential for 
the degree. Regular course testing and graded assignments have traditionally been presumed to 
account adequately for command of this knowledge. Indeed, several instructors may use the same 
examinations across many sections of the same course. However, in most media departments, core 
courses occur at the entry level (or lower-division) rather than senio'r level. Thus, having a core can 
simultaneously imply a standard body of knowledge but not be amenable to exit testing because the 
core courses generally are taken two or even four years before graduation. Both students and faculty 
perceive some unfairness about testing material coverel several years prior to the exam. Moreover, 
questions on introductory material do not approach what is meant by cumulative or comprehensive 
testing in the major. 

Moreover, there is wide disagreement on what constitutes an appropriate core.* Jannette 
Dates of Howard University, for example, argues for a grounding in history, ethics, and 
communication theory (Dates, 1990); others speak up for industry studies (Kamalipour, 1992) or 
production or a mix (Carroll, 1987). The 1987 Roper Report ("Electronic media career preparation 
study") fired up debate about the required and optional curriculum. However, a department with 
a set of core courses has at least taken the first step toward identifying a common body of 
knowledge. Whether this knowledge is, and even ought to be, retained in testable form until near 
graduation is an open question. 

Whether very large, diverse departments can or should have a well-defined core of courses 
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depends, at least in part, on whether they are within a "college of communication" or a "college of 
arts and sciences." In the first circumstance, they must replace a basic liberal arts curriculum, and 
a strong core consisting of several courses is an appropriate tool for structuring that learning 
experience. In the second situation, the curriculum largely serves as an upper-division major, and 
its very diversity limits its ability to require both breadth and depth. 
Observation 

Another method of finding out what and perhaps how much students are learning in the 
major is to directly observe classes. Although this can provide a realistic assessment of student 
ab.illtJ.^-,. iubest suits production and performance classes when the purpose is evaluating student 
learning. Observing in large lecture classes can provide considerable information about teaching and 
tell something about how much the students are paying attention and interacting but provides next 
to nothing about how much factual and conceptual knowledge they have gained. And of course, 
regular, systematic observation of all classes carries the pricetag of enormous demands for more of 
faculty's already limited time and energy. 
Peer Review 

External review by faculty from other institutions is widely used to assess the quality and 
viability of programs (see Schwalbe, 1991, for one report of assessment in a communication 
department). Typically, established senior faculty with expertise in the department's specialties and 
in program evaluation are invited to spend a couple of days reading reports, listing to presentations, 
and asking questions about a department and its faculty, curriculum, finances, and students, 
ultimately preparing a written assessment of the department's strengths and weaknesses, its 
reputation, and the field's likely future. Such assessments are useful in setting goals for departments 
and criteria for allocating resources to meet needs and goals, but tell little about what and how much 
students are learning. Such assessments depend on the assumption that an academically strong 
faculty, adequate facilities and budget, and a curriculum consistent with that of most similar 
programs, will provide an ideal learning environment, and thus learning will take place. Although 
these are reasonable, even practical, assumptions, they are a long way from precise measures of 
students' cumulative learning. 
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Rising Junior Exams 

One counterpart to cumulative examinations is the entrance exam, a test administered to 
second -semester sophomores or low juniors as a requirement of admittance to the program (called 
a "rising junior" test). Few four-year schools utilize such tests in mass communication-related 
departments, but many employ a cutoff-grade in either prerequisite or entry-level classes. Indiana 
University, for example, requires all majors and minors to achieve an average grade of C in two 
lower-division introductory courses covering the history, regulation, economics, programming, 
technology, and social effects of the electronic medJa.^ 
Pro ducti on ^ki li Tests 

Many departments measure the ability of their undergraduates to handle television and/or 
radio equipment. Called production or equipment tests, such tests are often administered in order 
to certify some students to utilize expensive video equipment away from the classroom or to permit 
certified students to operate a radio station without direct supervision. Although these are certainly 
a form of assessment, they are rarely applied to all students. 
Writing Skills Tests 

Another type of requirement necessitates passing a writing skills test administered by the 
department, sometimes taken before or during major studies or just prior to graduation (Meeske, 
1980). At schools such as San Francisco State University, the test becomes a requirement for upper- 
division status and thus enough school-time remains for remedial writing help before coursework 
is completed. As a graduation requirement, however, a writing test has little diagnostic value and 
must be considered an "exit exam" intended to uphold the honor of the department or university in 
the eyes of the external world by awarding degrees only to students with minimal writing skill. 
Internships 

Still another method of assessing student application and integration of learning is the 
industry internship. This provides qualitative rather than quantitative measures of student learning 
and focuses on the outcome of application in a limited situation (normally a few weeks with limited 
responsibility at one station). 
Portfolios/Resumes 

Requiring students to assemble a portfolio of their coursework, often accompanied by a 
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professional lesume, is an option for professionally-oriented departments. Portfolios can include 
demonstration video or audio tapes, student essays and reports, and copies (or photographs) of course 
project materials Programs in which most students concentrate on television production (or 
journalism) may find formal faculty evaluation of such a portfolio in an interview before graduation 
an appropriate measure of each student's achievement. The interview, usually with two or three 
faculty, can be utilized to help the student plan his or her professional future as well as evaluate the 
student's self-presentation in the interview and in the portfolio. 
Senior Seminars 

A caeslaos. course intended to help students synthesize preceding coursework and apply it 
to their own futures is often called a senior seminar. Generally limited to 25 or fewer students, a 
senior seminar typically hinges on a unifying theme such as professional ethics, or a contemporary 
event such as the quadrennial national elections, and themes may vary from year to year or instructor 
to instructor. Another approach to the capstone course is to require advance student projects, such 
as research studies or advertising campaigns, and usually include in-class presentations. 
Senior Theses 

Undergraduate student research projects can be formalized as senior theses. These may be 
mini-master's theses if students have sufficient methodological background, or they can be highly 
polished student papers, typically achieved after many individual conferences with the teacher and 
many rewrites. 
Exit interviews 

Asking majors questions at the time they graduate is useful to provide feedback on courses 
and faculty. Called exit interviews, written questionnaires or oral interviews with graduating seniors 
are useful for getting a sense of student perceptions and satisfactions — or misperceptions and 
dissatisfactions. But such exit reports do not provide measures of knowledge gained during 
coursework. In addition, gaining more than a nonrandom sampling of student opinion just before 
graduation is difficult since no penalty can be attached to noncompletion or nonattendance. Results 
tend to be more useful as tipoffs for departmental curricular or procedural problems, and, where 
positive about individual faculty, as ammunition for supporting tenure and promotion cases. 
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Senior Orais 

Oral examinations or interviews can be required in combination with any other outcome 
measure or given as the sole assessment tool (in addition to grades). Faculty generally find oral tests 
difficult to grade and tend to pass nearly all students, but one-on-one contact provides a chance to 
explore a student's strengths and weaknesses and share the faculty's assessment in a setting that will 
drive the message home. More typically, orals partake more of interviews than tests with the aim 
of aiding the student to define entry-level and long-term professional goals. 
BAM Certification 

In the earl y 198fls thft R esearch Committee of the Broadcast Education Association proposed 
two levels of voluntary testing on the procedures, content, and uses of audience research (Fletcher, 
1981, 1982). Certification was intended to assure broadcast managers and others in the industry that 
students who passed the test had the skills and knowledge necessary to become users and analysts of 
syndicated and custom audience ratings reports (level 1) or had the advanced research skills necessary 
to those who design, conduct, and report audience studies (Fletcher, 1982). The proposal was worked 
on for several years, but did not catch on for several reasons, among them that multischool pretesting 
demonstrated the huge, and perhaps irreconcilable, range in ways of teaching audience research, and 
also predictable resistance from both students and universities to having outsiders recertify 
achievements that already appeared to have been completed satisfactorily on student transcripts. The 
proposal was put aside pending more standardization of teaching and finally made moot by changes 
in attitudes toward audience ratings. 
Exit Examinations 

Finally, we come to content-intensive exit examinations in the major, generally 
comprehensive tests given just before graduation that must be passed to obtain the degree. At their 
best, exit examinations mix factual and conceptual knowledge with application questions and allow 
for imaginative and creative responses. At their best, such exit tests are essay-style and open-ended. 
In practice, most such tests employ machine-scored multiple-choice questions. Such exams have 
some appeal in academic fields with a stable body of knowledge-or in which there is general 
agreement on what must be learned and retained. 

In fields such as Biology, History, and English, for example, considerable experimentation 
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has been undertaken with some success in developing written tests that measure at least part of what 
undergraduates must be familiar with in order to justify a degree. The most respectable tests have 
a written essay component as well as electronically scored items, increasing their reliability but 
substantially jacking up the time and effort needed for analysis. One science professor noted that 
the cost depends on the goal: If the purpose is to evaluate the faculty's teaching, a random sample 
of students could be tested quite economically; if, however, the purpose is to evaluate individual 
students, then two concerns arise: low levels of faculty interest and administrative suspicion that 
outcomes testing magnifies problems not successes. 

To date, no natiatvaL«sJu4-4«as have been developed for four-year mass communication 
departments, and few local tests are perceived as successful by the faculties using them. Decisions 
to adopt exit testing depend on such factors as (1) the pressure applied to the department by 
administrative or legislative bodies and (2) the proposed testing's purpose (to measure students or 
faculty or curriculum). Crucial issues include what would be done with the results, whether tests 
would be standardized on common coursework or individualized, and whether passing such a test 
would be required for graduation. The remainder of this paper reports the results of a national study 
of mass communication departments and then synthesizes the findings in relation to current scholarly 
knowledge of major outcomes assessment. 

National Study of Mass Communication Departments 



Because rumors of widespread exit testing persists without hard evidence of the number of 
schools attempting such exams and their successes and failures, and because increasing numbers of 
media departments were feeling pressure to begin quantifiable major assessment, in fall 1992 a 
national mail survey was conducted of the 276 four-year schools that are institutional members of 
the Broadcast Education Association (BEA) with undergraduate mass communication programs.^^ 
Its purpose was to find out exactly which schools now utilize exit examinations. The BEA 
encompasses virtually all colleges and universities with extensive coursework or major fields of study 
in such areas as mass communication, radio-television (with or without film), telecommunications. 



15 



ERIC 



broadcasting, communication arts, many speech communication with a mass media component, and 
journalism (if the unit. is not oriented solely toward the print media).^^ 
Survey Method 

To identify colleges and universities using some form of exit exam, all 275 four-year 
institutional members listed in the annual BE A Directories of 1990-91, 1991-92, and 1992-93 were 
mailed a brief questionnaire. Cross-checking multiple directories allowed for schools that had failed 
to renew their membership in time for inclusion in the most recent (1992-93) directory. After 
defining (broadly) exit examinations, the questionnaire asked only if the department 
(division/program) were now usJjag.^!;vi,aftaed to use, or had formerly used any form of exit 
examination for majors. Exact department titles were also requested, and each questionnaire was 
coded to identify some schools for follow-up interviews. The survey form was kept very simple to 
encourage quick response and because academic experience suggested that very few schools would 
have such a test and those that did would vary widely in type and motivation. 
Interview Method 

Drawing on the mail survey results, researchers telephoned the department/division chair 
or head, or failing that the faculty member who was the BEA representative, at all departments 
responding that they utilized some sort of exit examination now or would soon in the future. Based 
on pilot interviews, researchers inquired about the type of exit assessment (exit test, interview, 
senior seminar, senior thesis, portfolio, and so on); whether the state or the university/college 
mandated major assessment and of what type; and how the results of the major assessment were 
utilized (to evaluate students or curriculum/teaching). If a test was reported, interviewers asked 
whether it was objective or essay style, what kinds of items it included (facts and concepts, 
applications, skills, affective questions), how it was developed, who revised it and graded it, when 
it was administered, how it was pretested and validated, and whether students had to pass it to 
graduate. A copy of the test, if there was one, and permission to quote from it were solicited. 
When tests once used were referred to, the interviewer asked why a test was no longer being used. 
When a test was presently being used, the interviewer asked if the faculty in the department 
generally satisfied with its effectiveness and appropriateness. Open-ended and informal telephone 
interviews were utilized because academic experience suggested that departments would vary widely 
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from one another and not fit easily codifiable patterns. 
Results of Mail Survey 

Altogether, 226 questionnaires were received from four-year BEA member departments, an 
82 percent response rate. Not surprisingly, by far the overwhelming number of schools did not use 
an exit test, had not used one in the past, and had no plans for using one in the future. Survey 
results, confirmed by telephone, revealed that 94 percent of departments did not use any kind of exit 
test in the mass communication major, and only 6% did (14 schools). Many schools, however, 
reported using some other form of major outcomes assessment, such as portfolios, equipment 
competency test, senior thesis, se.?^s<^-;>n>ki^r.^^ As Table 2 shows, only 4 percent of schools 
reported that they planned to adopt exit testing in the near future. Moreover, there were only 2 
reports of discontinued teots. 

Table 2 about here 

Results of Interviews 

From individual mterviews, two distinct approaches to exit testing in the major emerged. 
One approach addresses the question of whether the students collectively are learning through a 
comprehensive outcomes test, thus demonstrating, presumably, whether the department has been 
doing its job of teaching effectively; such testing is sometimes voluntary and passing is never tied 
to graduating. The other approach attempts to measure individual student achievement, and passing 
the test is usually tied to graduation for each student (at least once pretesting of the exam is 
completed). 

Of the 14 schools giving exits tests as of fall 1992, 10 departments reported giving written 
examinations that students need not pass or, in some cases, even take.^^ These are used to assess the 
curriculum and/or respond to state-mandated requirements for assessment in the major. Five 
departments give multiple-choice test% 1 gives a half objective/half essay test, 3 give essay tests (one 
a case study), and one is an oral. 

The Communication Department at the University of Tennessee at Chattanooga 
maintains a battery of multiple-choice questions based on its core courses and gives 
a 100-item test the day before final exams start. The test is voluntary and students 
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do not have to take it to graduate, but attendance is very good. Occasionally 
refreshments are provided and local radio personalities attend, giving the event a 
social character that draws students. After a pilot test, items were evaluated by a 
testing expert from the School of Education. The test provides empirical data on the 
department's teaching to show annual improvement, needed because the state has 
adopted a formula for funding based on performance assessment. The department 
chair also thought the testing helped to keep required courses in focus and had 
improved testing within courses. 

The Broadcasting Department at the University of Tennessee at Xaoxville also gives 
an annual 60-item comprehensive test in the major (all multiple-choice) to all seniors, 
again to demonstrate improvement to the state's Higher Education Commission! 
Students must take the test (as part of the capstone course) but passing is not 
required for graduation; students are told "not to study and do not take the test very 
seriously." The items were reviewed by the University's assessment office and faculty 
at other universities for validity and are not changed from year to year. It is 
presumed that test scores are us6*4e-se^npeve- media teaching across state-supported 
schools. 

The Communication Department at East Tennessee State University tests only on the 
two common courses in the major curriculum, an introductory course and a mass 
communication law course, and all students' scores are summarized for administrative 
use. For individual assessment, students complete a practicum or a senior project. 

Grambling State University in Louisiana gives a university-wide test developed at 
Princeton that tests English and general knowledge and is required for graduation. 
The Mass Communication Department also gives an objective test with items 
customized for each of tour concentrations and items on the common core. The test 
helps the faculty to keep track of each senior's progress. It is" not used to evaluate the 
department or the teaching, only aid the student. 

The Communications Department at Bethany College in West Virginia requires a 12- 
hour essay examination (broken up over three days), supplemented by a 1-hour oral. 
Grading is Pass/Fail, but passing is not requ.i*ed for graduation. Questions are 
changed annually. Exit testing in the major is required by the college, and the 
department chair reported liking such testing because it was a synthesizing learning 
experience that required students "to pull everything togetner," to unite and relate 
knowledge gained in seemingly disparate courses. 

The Department of Journalism & Radio-TV at Murray State University in Kentucky 
gives each student a take-home case-study test in the student's concentration area 
(such as advertising, public relations, production management). Papers are graded 
pass/fail on content and pass/fail on style, but the scores do not affect graduation. 
Instead, two faculty summarize all students' strengths and weaknf :3es in each area for 
the department's benefit in improving the curriculum. 

At Colorado State University's Speech Communication Department, the faculty gives 
an annual test on all areas of communication, including electronic media (as well as 
argumentation, rhetoric, and speech). Participation by students is voluntary, and the 
results are used to demonstrate to the state legislature what the department does— in 
response to a state assessment mandate. 

The Communications Department at the University of La Verne in California 
conducts an oral test and portfolio examination with each senior prior to graduation. 
The test is not formal enough to grade and is used mostly to allow students to 
demonstrate what they have learned, give feedback on their departmental 
experiences, and receive guidance on the portfolio and job preparation. 
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Black Hills State College in South Dakota gives a mostly objective test that students 
need not pass to graduate in response to pressure from South Dakota's state legislature 
and the university administration. The results are used to evaluate the department 
and Its teaching. The Mass Communication faculty makes it up and gives it every fall 
and spring. Media studies is part of a comprehensive unit encompassing Speech, 
Theatre, Modern Languages, Art, and Mass Communication. 

Augusta College in Georgia gives its students in the Broadcast/Film Department an- 
internally-generated half objective/half essay test which includes both track-specific 
Items and general communication skill questions. Students are required to take the 
test, but it need not be passed to graduate. The test serves as an advising and 
curriculum tool when a pattern of deficiencies emerges. The test is given at the end 
of the fall quarter within the senior seminar and graded by the main faculty in each 
area. The test was instituted by the department. 



Of the 14 aggregate schools presently v5m3-2-n- 9^:4t-test, only 4 give written examinations that 
majors must pass to graduate. One is an objective-style multiple-choice exam, and three are essay 
examinations. 



Like all other departments at the University of Tennessee at Martin, the Communi- 
cations Department requires its majors to pass an objective-style test. The University 
also requires a pretest/posttest on general education given in the freshman and senior 
years. If an increase in the average scores is shown, the University qualifies for the 
state's performance funding. Only a posttest is required for departmental majors, 
and Communications gives it within its senior seminar. Tfie test is intended as an 
inventory of common (50%) and specialized (50%) coursework. The core consists of 
seven courses (21 credits) and represents half the major; the specialized tracks are 
Broadcasting, Journalism, and Public Relations. In addition to using the test as a 
requirement for graduation, the faculty averages the students' numerical scores to 
track the department's progress from year to year. The chair reported that the 
faculty was generally dissatisfied with the test and considering supplementing it with 
a portfolio evaluated by the faculty. 

At the University of Arkansas, Little Rock, the Radio-TV Department has just begun 
exit testing using an essay exam. The faculty negotiated ten concepts and questions 
based on them to form a pool from which exam questions are drawn. The test is 
grade pass/fail by five faculty. One special value of the concept-pool approach to 
the department is its guidance for part-time faculty. It has also helped to standardize 
the content of the department's seven required courses. This assessment process was 
developed in proactive response to indirect pressure from deans and accrediting 
bodies, in part to forestall more rigid state requirements. Such testing is 
acknowledged to involve a lot of extra work. 

Kutztown University in Pennsylvania conducts an essay test within a required senior 
seminar (graded by the course professor but constructed by all faculty and changed 
every semester). The test deals with the major subject areas of production and law 
and IS graded pass/fail. It is given every semester and summers. At least some of the 
faculty are "not thrilled now" by the test. 

Xavier University of Louisiana utilizes a two-part test in the Communications major, 
the first part of which is an essay test and the second part a television equipment 
practicum requiring problem-solving (it includes editing a news story). The essay 
test Itself divides into comprehensive questions on mass communications (80%) and 
focused questions on core courses (20%). The 3-hour written essay portion is given 
on a Saturday in spring and fall, and it is graded pass/fail. The faculty claims to 
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have learned a great deal about the program's strengths and weaknesses from these 
ests and are now revising the curriculum. They are, however, considering reducing 
the essay test to only 50 percent of the exit requirement and adopting a two-step 
portfolio/resume requirement: each student would initially prepare a portfolio at the 
sophomore level and then revise it at the senior level to show progress. 

In addition to the 14 schools already described, 8 schools reported that they planned to adopt 
an exit test. Most have opted for multiple-choice-style exams, but only 3 are sufficiently far 
advanced to be described here. 

At San Jose State University, the Radio-TV-Film area (which is part of a larger 
department that includes Dance and Drama) has been preparing for the last two years 
a comprehensive multiple-choice test to be administered in one of four senior-level 
required courses. The test will cover a.lUh^e^a.rt-ment's major broadcasting areas 
(corporate video, production, writing, and social/historical/political communication), 
but at east initially, students will be required to take the test but not to pass it to 
graduate. The department is also weighing the possibility of administering the test 
at the rising junior level so as to have both pretest and posttest results Having 
outcome assessment was mandated by the University's trustees, in response to state 
egislative initiatives, accreditation association pressure, and grade inflation, but the 

a^e'adopdnTrtfoS'' °" ^^^"^^ ^"^ ^^^'"^' -^^"P'«' 

The Communication Department at Sangamon State University in Illinois is also 
planning to institute a multiple-choice style pretest/posttest to provide empirical data 
on what Its students learn. Separate tests will be developed ih the department's three 

svstlm°<f T'lr.fL'J'K'^K'' ""^^"'"^ '^i'^""'' ^"^ interpersonal & organizational 
systems. At least at the beginning, students will not have to pass to the test to 
graduate, but it is expected that the data may eventually affect salaries and other 
faculty concerns down the road. Quantitative assessment in the major in Illino s is 
being driven by legislative budget reductions and threats to cut entire departmen! " 
to save money. The University mandated quantitative outcomes assessment for all 
major programs to stave off these cuts. ^miiciu lor au 

At Southeastern Louisiana University, the Communication and Theatre Department 
IS planning an objective-style test as part of a three-prong outcomes assessment 
^rZTi"^'"^ 'ncludes (1) skills testing (covering talking and reading, perhap Tn a 
random y selected subset of students), (2) attitudinal questions ask ng about the 
curriculum and courses, and (3) cognitive test items on content in the major but 

cTuS'Se'^-n h '^^'.'^^^.'^^ (-hich include one mass commuiic^t'on 
course). There will be no questions in production or any other specialty The test 

rt'mSL7.hnnU ^ ''-""^ ^'1! '"^"'^"^ '° Assessment here, as 

at other schools in Louisiana, is being driven by the accreditation process. 

Several other schools, including West Texas State University, the University of Southwestern 
Louisiana, Jacksonville State University in Alabama, Amarillo College in Texas and the University 
of Central Oklahoma, have less definite plans for exit testing, in 3 cases for multiple-choice tests, 
and in 2 cases for essay or part-essay tests. 

Only two schools reported having once had a exit test that they no longer have. The 
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Communications Department at De Pauw University in Indiana had u test about ten years ago. It 
involved a Saturday-morning written component and an oral compc r.ent, but it was dropped because 
it took too many faculty to manage it as the number of majors grew from 30 to 150. The 
Communications Department of Loyola University of New Orleans also reported having a multiple- 
choice exit test until 5 years ago. At that time, the University required an exit test in each major, 
but the departments were not allowed to flunk anyone. It was dropped when cooperation on the part 
of students and faculty ebbed. 
Alternatives 

Some schools have come down on the side of ncRqu^i-ntits-tiva outcomes assessment. At least 
four schools, among them Eastern Connecticut State University and University of North Dakota, 
plan to adopt portfolios as an outcome assessment tool (in addition, several schools are already using 
them); Southern Illinois at Edwardsviiie plans to assess majors by means of a major paper on their 
learning and internship; others with a production orientation utilize an equipment competency test, 
sometimes combined with a content test or a project or a portfolio. Still others have adopted a 
senior project, either in the research or applied arena, as at Waiia Walla College, Southern Illinois 
University at Carbondale, Cedarville College, De Pauw University, or La Sierra University. Many 
schools the researchers talked with, however, were still weighing their options. 
Predictions 

Some schools still had a year or two to get an assessment plan in place. In Oklahoma, for 
example, the class of 1996 will be the first to undergo testing on their four years of education. In 
Detroit, however, Wayne State University administration has insisted that all departments begin 
assessment in the major by the end of 1992-93. Although Louisiana schools are under strong 
pressure to begin outcomes assessment as part of the accreditation process, one chair claimed that 
"severe budgetary problems-to the point of financial exigency-will forestall action." Other chairs 
felt the opposite way: Greater pressure on legislators to divide up a shrinking pie of state funding 
will lead to increased insistence on outcomes assessment to justify allocations. 

Several chairs reported their faculties as "very resistant" to outcomes assessment in the major. 
One typical comment was, "We would like to have a comprehensive test and oral, but my faculty 
says 'Where's the time?'!" Another chair pointed out, "If a student completes a curriculum designed 
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to prepare him or her for a particular profession, I don't think giving a test later (or a practicum 
course) can be any more effective as an indication of ability and achievement than the grades already 
given in classes." One respondent from a North Carolina school was outraged: "Such exams are not 
used on this campus in any way. I can't imagine our administration allowing such, much less 
encouraging it" [emphasis his]. Another South Carolina respondent agreed: "Winthrop University has 
been and is a leader in assessment in South Carolina and the Southeast for several years; one firm 
principle we have is 'NO EXIT EXAMS,' although a few departments have unwisely attempted 
them." 

Another chair said, "We're facing a general expectation- in the- university; there is no 
requirement, but we're going to have to be accountable." Another said, "The Regents are mainly 
interested in posttests, in evaluating the effectiveness of the university." Higher administration are 
attempting to find means of proving that they do a good job, or if they don't do as well as expected, 
attributing the cause to lack of money. Department chairs want positive data that are reasonably 
credible to help them make a case; they don't really want, in most cases, evidence of weakness. Thus 
departments might consider putting aside their own need for curricu'lar feedback in favor of testing 
student achievement, nut most faculty are rather idealistic; they will accept useful measurement tools 
administered credibly that give them feedback. They will be unable to accept pro-forma testing that 
measures nothing but that can be manipulated to show ever-increasing total scores. 

The experience of Loyola University of New Orleans warns us that faculty and students may 
eventually rebel against tests that are not required for graduation if other reinforcement is not 
forthcoming. The experience of Eastern Tennessee State in turning a not-required test into a social 
occasion suggests one happy option. 

The issue of law suits bothers some faculty. The chair of the Department of Mass 
Communication at Bemidji State University raised these questions on behalf of his faculty: 

"Would a student be denied graduation if he/she failed the exit exam? What would 
It say about the quality of instruction if a student didn't pass— especially a last- 
quarter senior? What action would be taken if the student didn't pass"? Could the 
student file suit against the university?" 

He pointed out that Bemidji was working instead to stiffen entrance requirements, to screen students 
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before they come into the program. Many departments do that, of course, and find it useful, but 
entrance exams do not address the same issues as outcomes assessment. 
Analysis of Respondents' Examinations 

Ten departments currently using an exit examination shared a copy of the test with the 
research.^® Half of the exit exams were objective and half were essay/short answer. The objective- 
style tests ranged from a low of 60 to a high of 150 multiple-choice/true-false items, but typically 
had 100 items. One short-answer/definitions test focused solely on audio and video production; 
another essay tests called "comprehensive" focused solely on an internship experience. Three 
additional essay tests varied from analysis of a single case study to a dozen short essays. 

The sample of five objective and five essay/short-answer tests is too small for any 
quantitative analysis, but a few comments from students seem appropriate. A small honors class was 
asked to examine the tests and rate each one for scope, difficulty, and globally for fairness. 

First, the quality of the objective questions was generally weak. These students found four 
of the five quantitative tests too easy: three 100-item multiple-choice tests because they covered too 
much territory too superficially, another specialty-only test too limited to be sufficiently difficult. 
They likened the questions in the comprehensive [read broad] tests to those used in large, lower- 
division survey classes (such as those using Head & Sterling's Broadcasting in America or Dominick's 
Broadcasting /Cable and Beyond) and felt that this was inappropriate knowledge to test on several 
years later. On these four tests, they judged many questions poorly constructed (and testing experts 
would agree). The fifth quantitative test was thought to involve too much professor-specific or 
book-specific memorization and too little analytic thinking, but it was judged significantly better 
than the others. However, it remained a test of introductory material, not a student's specialization. 

Here are some examples of elementary, trivial, and poorly-constructed multiple-choice items 
drawn from several schools' tests. Dozens of questions were pointed out as general knowledge 
information (at least among young people) or of the "easy guessing" type that required no information 
learned in school. Surprisingly, one school made it clear that some questions on a graduation test 
came from a course entitled "Introduction to Mass Communication." Questions #1-3 are the types 
considered too basic (in media studies) for graduation testing: 

1. The signal for VHF, UHF, and FM 
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a. is sent in digital mode. 

b. travels by microwave. 

c. travels primarily through the ground. 

d. bounces off the ionosphere. 

e. travels in a straight line. 

2. The largest share of advertising revenue goes to 

a. radio. 

b. newspapers. 

c. television. 

d. magazines. 

e. direct mail. 



3. The Sullivan decision provided 

a. a definition of libel. 

b. a definition of malice. 

c. a definition of negligence. 

d. a definition of a public figure. 

e. a distinction between public figures and private persons. 

Questions #4-6 are were evaluated as too trivial for graduation testing. Question #4 was pointed out 
as the sort of general knowledge that doesn't belong on any test. Questions #5 and #6 were noted 
as simply unimportant information. 



4. An album that has sold 500,000 copies and a single record that has 
sold one million is considered 

a. gold 

b. platinum 

c. silver 

d. brass 

5. Appropriation consists of 

a. stealing another person's money. 

b. using a person's name, reputation, or likeness for commercial 

purposes without consent. 

c. destroying a person's reputation. 

d. trespassing on someone else;s property in order to cover the news. 

e. writing a news story about a person without permission. 

6. Harlem Book of the Dead is by 

a. James Van der Zee 

b. Avedon 

c. Gordon Parks 

d. Karsh 

Questions #7-9 were pointed out as examples of exceptionally poor questions. Question #7 is of the 
type which all students would answer V to because no other answer is viable. Questions #8 and #9 
were considered "insulting" test items that underestimate young people. Testing experts would also 
point out that five full answers should be included for each question to reduce the odd for guessing 
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"•nd that "j».U of the above" and its counterpart "none of the above" are to be avoided: 



7. Theory Z of management assumers that employees are 

a. to participate fully in the running of the company^ 

b. not to be trusted under any conditions. 

c. not creative and have to be heavily supervised. 

d. to be paid bare minimum. 

8. Nonverbal communication consists of 

a. sign language 

b. hand waving 

c. smiles 

d. all of the above 

9. Printed or written defamatory material is 

a. gossip 

b. slander 

c. libel 

d. none of the above 



An example of a test questions they found fair, of sufficient difficulty, and requiring either 
important factual knowledge or analytic thinkging is in Question #10. 



10. The media of this nation increasingly are owned by large corporations rather than 
independent owners. Many view this trend as a threat to 

a. the law of large numbers 

b. equal access rules 

c. a free marketplace of ideas 

d. the fairness doctrine 

e. recent FCC rulings on exclusivity 



Second, the honors students found marked bias toward different subject areas even in tests 
appearing to be "comprehensive" One lOO-item test, for example, favored regulation and policy (55 
items) over all other subject areas (45 items). Another favored public relations even within the "core" 
portion of the questionnaire. Others tended to favor the history of the field. 

Third, the evaluating students preferred specialty tests over basic industry information. They 
thought tests given at graduation time should concentrate on material learned in upper-division 
specialty classes, not lower-division introductory material. They thought testing library skills (how 
to interpret a bibliographic reference) at graduation was rather useless (it belonged at a much lower 
level), but they approved of English language proficiency segments in tests even at the senior level. 
Ito.ms on grammar, word choice, and correct usage were approved. 

And, not surprisingly, these honors students uniformly thought the essay tests (or essay items 
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at the end of mixed tests), requiring application to "real world" situations, to be the best measures 
of what students had learned. However, even these test items varied from superficial to meaningful 
in the students' view. They liked best case study questions such as the following, but most case 
studies (in the sample sent to us) proved to relate to news journalism (even when they were called 
"public relations" or "radio/television" questions). 

1 1 . The editors of [student newspaper] would like to improve the paper in terms of the stories 
It covers and its layout and graphics. Before doing that, they are eager to know what 
students thmk about the paper and what they would like to see included in [newspaper 
name] m the future. You volunteered to do a scientific survey of student opinions. 
Describe in detail the steps you would take to determine the students' opinions. 

12. All journalists, broadcasters, public relations consultants, and advertising executives are 
faced with ethical decisions daily. Review the differing ethical principles that may arise. 
Remember to link the ethical principles very explicitly to your examples. Accurate 
illustration of the principles you raise should be your main aim. 

In sum, these honors students judged most multiple-choice questions as tests of test-taking 
skills rather than accumulated knowledge and ability to integrate and synthesize. They found most 
objective test items too basic, often trivial, and too often poorly constructed. Short answer/essay 
questions were usually better for demonstrating writing and thinking skills, but they were also 
unsatisfactory (in terms of scope, level, and fairness) when they measured memorized subject matter. 

In my view, exit testing, if it must occur, should focus on students' writing, researc'h, and 
analytic skills, not memorized material. Individualized or small-group tests should be constructed 
that match the subject area of each student's interests insofar as possible, rather than using the same 
test for all students. Differences in subject matter are irrelevant to measurement (but important f. - 
motivating students to do their best). Evaluation should focus on the student's ability to apply 
knowledge and skills and avoid rote learning of facts and measurments of objective test-taking skills. 
Recommended Model 

At its start, this paper promised a model for action. To develop one, the author drew on what 
other schools are doing and plan to do and what other faculty, as well as the author, think ought to 
be done. This model is phrased as a series of sometimes controversial statements. 

1. Whether to test depend.s on resources available. 
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Small departments with 10 or 20 graduating seniors may be able to muster sufficient resources, I do 
not say energy, to write, administer, and grade individually customized tests. They can take account 
of the specific courses each individual has completed and design measures that allow students to 
show their strengths as well as weaknesses. For such departments, essay-type tests, oral 
examinations, and portfolios of work can get students to display their ability to integrate, synthesize, 
and apply their coursework. Large departments, graduating anywhere from 50 to 200 or more 
students annually, on the other hand, lack the faculty time and commitment for anything but 
objective-style tests, and since these seem to test the wrong things, they should avoid exit testing 
entirely. 

2. How to assess depends on whether the goal is preparing students or measuring students. 

In such states as Tennessee, Oklahoma, and Illinois, legislators now require plans for quantifiable 
measurements of teaching/learning, commonly called outcomes assessment, but faculty have some 
leeway. What becomes crucial in deciding what kind of test to implement is the department's goal: 
Is the purpose of assessment (1) to measure the individual students''achievements or (2) to measure 
the department's achievements? 

If the goal is to measure individual student achievement as a precursor to granting a degree, 
tests often are the poorest option. Tests must, by their very nature, be standardized, and if required 
for graduation, often must be quantifiable and legally defensible. Yet educators recognize that 
multiple-choice tests are weak tools for measuring anything but the most basic factual material. 
They rarely succeed in measuring students' ability to integrate their learning, synthesize seemingly 
unrelated ideas, apply skills and knowledge in new situations, theorize about causes or impacts, 
demonstrate analytic or creative thinking, and so on. Indeed, objective tests tend to devalue what 
is important about education— the creative, controversial, and immeasurable— while fostering what 
is trivial, valueless, and measurable. Interviews with departments required to assess their majors 
reveal widespread adoption of several alternatives to testing, including senior seminar, senior thesis 
or project, and portfolio/resume. These options appears more credible and efficacious measuring 
the individual student's achievement in the major. 

If the goal is to assess the department's achievements in teaching, a test may be appropriate. 
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As the interviews show, testing students on content learned in the major may be initially useful in 
providing feedback to the faculty about weaknesses and strengths of coursework, curriculum, and 
teaching. The results of tests can be compared over time to demonstrate improvements in 
curriculum. However, on the negative side, eventually such testing can have an undesirable impact 
on courses as faculty quite naturally begin "teaching to the test." Moreover, in small departments 
where who teaches what course is well known, the results of such testing can readily have impact on 
merit evaluations and salary increases. Such a danger is less apparent in large departments where 
several individuals teach the same course. 



3. If testing is for the department's benefit, test at intervals. 

One option for avoiding many of the negative side-effects of testing is to not test every year. 
Testing majors at 3- or 5-year intervals minimizes the likelihood of the curriculum being driven by 
the test and individual faculty salaries being tied to test results. Furthermore, testing with the goal 
of assessing the department need not be required of all students; large departments can test random 
samples, provided they have sufficient coursework in common. 'Alternatively, the faculty can 
develop tests only for those in a specific emphasis or track, and vary the track tested on each year. 

4. If testing Is used, incentives are needed to get both students and faculty to take the test seriously. 
Faculty need explicit rewards for devoting more of their scarce time and energy to assessment, 
beyond what they already do. Students need immediate rewards for participating in assessment, and 
purely negative "rewards" (such as "you will not graduate if you don't pass") are the educational 
literature teaches, wholly undesirable and will have long-term negative effects on the students' (and 
thus the public's) view of college. Indeed, academia may already be suffering from this problem. 

5. If testing Is used, examine students on what they have studied recently in their specialties. 
Some faculty presume that, to be fair, a test must concentrate on what students have in common, 
the department's required or core coursework. However, in many departments, most required courses 
are completed in lower division, sometimes as freshmen; it is hardly appropriate educationally to test 
on that material two or four years later, nor would such a test tell educators anything they want to 
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know. 



6. Avoid the pretest/posttest trap. 

Too many academics, facing too many demands and pressures, see the pre/posttest paradigm as a way 
out of the "testing to show improvement* dilemma. However, not only will such tests reveal nothing 
of value, their content will creep insidiously into the curriculum, eventually resulting in the worst 
sort of "teaching to the test." For the most cynical faculties, such tests can easily be manipulated in 
high unethical fashion to demonstrate so-called improvement. 

7. Choose a test based on the department's curricular goal($). 

If the department's central mission is to prepare its students for graduate study, then utilizing the 
general section of the Graduate Record Exam (GRE) or Michigan State's test for entering graduate 
students makes some sense. If the mission is to prepare students for the broadcasting workforce, then 
portfolios make sense. If the mission is broad and the curriculum will have varied outcomes for 
different groups of students, then no single tool is appropriate. 

8. Puncture the myth of the nationally normed test. 

Media departments are too diverse in orientation and goals to ever have national standardized 
outcomes tests. Even statewide, departments do and should vary to reflect ;he specialties of the 
faculty. Not all schools need teach the same things; indeed, fostering clones by standardizing mass 
communication curricula across schools wastes precious resources and fails to make the full range 
of options available to students. Moreover, nationally normed tests apparently are weak tools for all 
fields, even the sciences. As knowledge has proliferated, so has specialization in curricula across 
the entire university. 

9. Insist that legislators pay the cost of developing local measurement tools. 

Even within one large department, students take a wide diversity of programs of study and the facts 
and concepts of the field change with unparalleled frequency. In addition, any test must be 
constantly rewritten to avoid breaches of security, and multiple-choice test items should have 
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evaluation by testing experts to avoid charges of bias. Testing must be viewed in light of pressures 
for multiculturalism. The sum total of (1) customization to meet individual student needs, (2) 
reevaluation of test items to keep pace with industry changes, and (3) development and testing of 
new questions will have a prohibitively high cost. Most media departments are understaffed and 
underfunded while overrun with students. They must have financial help before taking on new 
tasks. 



10. Consider the underlying purpose of undergraduate education before buying into the idea of 
assessment. 

Is the learning of content the goal of undergraduate education or is it demonstrating the ability to 
learn? Does the faculty aim to pour concepts and facts into the receptacle of students' minds? Or 
are they much more concerned about imparting the reading, writing, investigating, and thinking 
skills essential to learning through the student's professional and personal life? If you come down, 
as I do, on the side of learning how to learn as the central purpose of undergraduate education, then 
assessing cumulative knowledge seems profoundly counterproductive and a wicked waste of scarce 
resources. Indeed, even a bit surreal . . . 

Do not misunderstand. I am not opposed to assessment. Indeed, we assess students all the 
time and should do it more efficiently and effectively. I am opposed to objective tests that foster 
a misapprehension about what education js and is not . I have a great horror of giving in to those 
legislators, trustees, higher-education officials, and even colleagues who want to reduce education 
to something simple, an in/out process. For me, and I hope for those reading this paper, education 
is an ongoing process of growth and expansion, fostering the abilities of learning how to learn, 
responding creatively to one's environment, thinking, imagining . . . Especially at the 
undergraduate level we must resist the narrow idea that our job is to prepare students for the job 
market. What that job market will be tomorrow is unknown, and what the rest of our students' lives 
will be is unknowable. 
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Table I 

Disadvantages of Common Testing Methods* 



Objective 
1 - Limited coverage of content 

2. Test construction costly 

3. Security concerns require 
frequent rewriting of exam 

Based on Ervin, 1988, p. 21. 



Essay 

1. Hard to agree on guidelines 
for scoring 

2. Enforcement of guidelines 
nearly impractical 

3. Extended time required for 
grading 
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Table 2 

Results of Survey of Four- Year BEA Member-Schools 



No.of Schools 

DO NOT USE AN EXIT TEST: 212 
Discontinued a former test: 2 
Planning to adopt a test: 8 

PRESENTLY USE AN EXIT TEST: 14 

DID NOT RESPOND: 50 

N = 276 
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Endnotes 



1. For institutions of higher education, the term assessment nowadays refers to ways of 
evaluating an institution's ability to accomplish its purposes. The original referent of formal 
A iTAS '^^f" ^"''^"■^'^d in the broader concept of accountability. Accord inTto the A^r ca 
Association of University Professors, "The present assessment movement is in part a response to 
increased demands for public accountability and in part a byproduct of variournationaTreno^^^^ 
on the status of higher education in the late 1980s which criticized both groSg ?Sarch 
emphases in the nation's colleges and universities and the quality of undergra^ate education " 
^cfli/wi^, November-December 1990, p. 35. """cigrduudie eaucation. 



a'bTner^nH^rcf V ^/^"^•"'.st'-.afoi-s colleagues in colleges of communication generally have 
a better understanding of the limitations and pressures under which media faculty work and thus 

S^nf." ^'■'■'y- ^^"^"y ^ --^'^^^^^'y ^^^^^^^^y acknowledged deparLent W 

Z^TJ r ?' T'^-" ^^P^::^"^^"^^ typically operate under enornroXB prAurfto acTommS 
more studems re ative to the size of faculty and facilities than older departments cSla are 

fnde"r.\l'?:?H'''"'''' ""T-' '^'^^"S'"^' '^"^ing themseWes to easy 

understanding by more traditional faculty; media faculty's research and teaching is usually widely 

oi acnievements ditticuit to achieve; moreover, media attracts relatively little bv way of erant 
money compared to the hard and social sciences. But typically, the larger traditional 
departments can best spare the faculty to sit on tenure and prLotion comm «ee f"^^^^ are rh. 

?acuYtlT'' Tr""' '^'"^ '^^'^ '^^"^' ^^^""^^"^ in the ma^r^ rTesSe hreat to 

faculty in media departments than in the traditional fields. 

?n '^^^V ;«lPO"S'.'''''ty center management (and RCB to responsibility center budeetine) 

L^^VI; V ' of the university are designated "responsibility centers" and cLtro their own 
Sfw- °" other funds) and expenses (salaries and facilities). In prac?i^e toth la^ge 
units with large budgets and those generating outside income in the form of grams' and Datent 
royalties tend to be in a better position than either small units or those withou t ou side reveJ' 
(where electronic media departments typically fall). outsiae revenues 

4 This arises in part from the national visibility of the American Council on Education in 
Journalism and Mass Communication (ACEJMC), a body that sets sSa ds for anr.^/r.H ^c 

• iZ7r:7 '''' for confmun cation r d o-tl^^^^^^^^ 

broadcasting, or telecommunications departments. television, 

5. Ervin surveyed colleges and universities listed in the 1987 Journalism and Mass 
romm!!n>'l'°" directory published by the Association for Education in JournaHsm and Mass 
fhrB=°Edu^caS?^^^^^^^^ ~S ^n ^^arf 
3[\o^£rsfg= -P'e'tiS^Kg a^ft ^p^rrenf.;r t?o"°^ 

6. Members of the Broadcast Education Association include such diverse orientations a<= 
Instructional Media Communication Studies, Science and TechLlo^y SfvSon Rad^^^^ 
Television/Photography, Theatre and Communication Arts, English and Communicadon 

and Et"cfrnn> ^T'^''' •^"^.Spee'^h Communication/Theatre Arts/Communication Disorders 
and Electronic Media Division in the College/Conservatory of Music. uisoraers, 

7. For most media faculty, this triggers thoughts of ACEJMC accreditation the form of stanHarH 
curriculum with which media faculty are most familiar. ACEJMC, for Sampfe limits^^ 
nuniber of hours students can take in the journalism major. Other than the reladveTy b Sad 
fSd'h^rnn '"'fi'r' ^^^^^''^.^^^ professional associations, no academic body S'the media 
field has attempted to set curricular guidelines. Occasional scares such as the 1987 RoperRe port 
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precipitate months of self analysis (mostly self justification) but have not, to date, ended in field- 
wide curncular guidelines. 

8. A look at college catalogs immediately confirms this view. However, according to a recent 
survey by Robinson and Kamalipour (1991), there is considerable commonality in curricula. See 
also Porter & Szolka (1991) for comments on liberal arts versus professional preparation. 

9. A variant on the rising junior exam at a four-year college is the cumulative exit examination 
given at the end of two years at a community or two-year college. Such a test measures 
introductory material and what might be gained in a two-year major program. These tests need 
not tace the immense diversity of individual programs that characterize four-year curricula. 

10. After two years of investigation, however, Indiana University's Biology Department dropped 
plans for such testing, largely on the grounds of its insupportable cost and hidden hazards 
Faculty commented that the costs of testing would equal the cost of hiring three or four new 
faculty, and, were the funds available, the benefits from smaller classes would far outweigh the 
benefits of testing all near-graduates in a program with about 1200 majors. Faculty also noted 
that even m a well-defined science such as biology, nationally normed tests were not tailored to 
he curricula of specific departments. Administrators were also wary of testing as it seems more 
hkeiy to make departmental weaknesses salient than parade departmental strengths For this 
biology department, it was found that more careful analysis and comparison of final course exams 
in a single, key required course was more profitable for the faculty and the students. 

11. Three schools, Columbia School of Journalism, Regent University, and Ohio University 
bcripps School of Journalism, were not included because they have only graduate programs; the 
University of Maryland was not included since its program has been dropped; William Jewell 
college was not included because its Communication Department has no media-related track. 

12. The BEA Directory was mo. e appropriate as a sampling frame than the Mass Communication 
Division mailing lists of either the ICA or SCA because they attract only the more research- 
oriented faculties. Similarly, the AEJMC directory includes too many schools with only a print 
orientation, whereas most departments taking a broader look at mass communications would 
probably belong to the Broadcast Education Association. 

13. Questionnaires were addressed to the department chair by name, if listed in the BEA 
Directory; otherwise the questionnaire was directed to the BEA representative by 



name. 



14 These reports were not quantified as the information was volunteered on a questionnaire 
asking only about exit testing. 

15. A very few schools reported that their colleges or universities already had an exit examination 
in general eaucation. Franklin Pierce College, for example, administers an exit test in English 
and mathematics but not in major fields. 

16 Two additional tests assessed the department and curriculum, not subject matter, so they are 
no considered here. Several schools also sent copies of their senior seminar syllabi to be shared 
with interested colleagues. 
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